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instruction and research. The indexing system consists of an on-line 
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searching, and a programmed text for teaching index preparation. The 
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THE USE OF AN ON-LINE SEARCHED AND PRINTED 
COORDINATE INDEX IN TEACHING 

G. Oahoda and Ferol A, Foos 
ABSTRACT 

An indexing system to 1850 documents on library automation* 
systems studies in libraries, and indexing systems was developed 
for use in instruction and research. The indexing system consists of 
an on-line searched coordinate index, a printed coordinate index, a 
subject authority list, an abstract bulletin containing the 1850 
documents in the index, computer-assisted instruction for index pre- 
paration and searching, and a programmed text for teaching index pre- 
paration, The system has been used for instruction of graduate library 
school students in index preparation, searching, and evaluation. 
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THE USE OF AN ON-LINE SEARCHED COORDINATE INDEX 
IN TEACHING 

G, Jahoda and Ferol A, Foos 
I, Introduction 

The objectives of this project are the development of an on-line 
searchable coordinate index called FOCUS (£ lorida State University On- 
line Coordinate Index Use Study) and its use in teaching and research, 

The project is carried out in the following stages: 

1, Planning of indexing system; 

2, Preparation of the first version of the index and 
its use in teaching; 

3, Enlargement and refinement of the index and preparation 
of additional instructional material; 

4, Use of the enlarged and refined index in teaching; and 

5, Use of the enlarged and refined index in research. 

In this report, the present state of the- index- will be summarized 
and the work carried out between September 1970 and May 1971 will be 
reported. This represents stages-three ana four of the project. Stages 
one and two have been reported (Jahoda A Foos, 1970) and will be 
summarized as needed for background, 

FOCUS is centered around a decision making mode! of index 
preparation and use, The model which is depicted in Figure 1, has 
been used in classroom instruction, in computer-aided instruction, and 
in a programmed text, 

The index now consists of about 1850 documents on library automati 
systems studies in libraries, and indexing. The equipment used at the 
Computer-Assisted Instruction Center (CAI) is an IBM 1500 Instructional 
System consisting of an 1800 central processor with 32,000 words (16 bits) 
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of core, a 1502 station controller, sixteen 1510 CRT displays each 
with a keyboard and a light pen, one 1518 typewriter, and five 
disk drives with removable disk packs of 512,000 16-bit words (1,024 
million bytes). 

FOCUS now consists of? 

1, an on-line computer searched- coordinate index' 

2, a printed coordinate index; 

3, an abstract bulletin containing the indexed documents; 

4, a subject authority list (Sal); 

5, a computer-aided instruction program for index 
searching; 

6, a computer-aided instruction program for index preparation; and 

7, a programmed text for index preparation. 
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Figure 1 . ""Coordinate Index Searching and Preparation as Decision Making Steps 
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THE FOCUS INDEXING SYSTEM 

The initial coordinate index to 700 documents* whose development 
is described in Tech Memo No-, 22, had the following- characteristics: 

1, indexing based on abstracts; 

2, indexing by subject, date of document, and- type of document; 

3. indexing as specifically as subject of document permits; 

4. indexing on one level only; 

5, controlled vocabulary by means of a subject authority list (Sal); and 

6. no roles or links. 

The testing of this index by a class of library school students 
brought out several deficiencies. Between June and September 1970 , 
identified deficiencies were corrected and the indexed collection was 
increased from 700 to I860 documents. The basic characteristics listed 
above were retained and are still applicable, although the following 
changes were made in the index at that time: 

Word variants ,- Descriptors such as Editing, Editions, Editors, 

Files, Filing, and Verification, Verifiers were examined to determine 
whether they should be kept as separate index units or combined; and, if 
kept as separate descriptors, whether they should- be connected by cross- 
references. 

Scope notes .- Scope notes were added to more descriptors, e,g,, 
Planning, Information Analysis Centers, Space, to delineate their meaning 
in the index. The six types of document descriptors (Bibliographies, 

Case histories, Philosophy* Research, Reviews* and Surveys •)• were redefined 
in expanded scope notes to increase consistency of use by both searchers 
and indexers. 
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Hierarchical- list,- A hierarehical arrangement' ef descriptor 
was prepared as an aid in searching. The- revised -edition of the Sal 

Includes the hierarchical -list along- with- the- alphabetic list of 
descriptors , 
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INDEX SEARCHING PROCEDURES 

Instructions for on-line searching of FOCUS are given in Figure 
2, Search statements can be made for as many as 20 descriptors combined 
with logical '-‘and, 1 "or, ;: "not* connectors » The- initial computer 
response is a display of- the number -of potentially- relevant documents. 

The searcher has two separate options; he can ask the computer to 
display either the full bibliographic citation or he can specify the display 
of only the accession numbers of the potentially relevant documents. He 
may also renegotiate the search if either too many- or not enough potentially 
relevant documents result from the specified search strategy, 

The expansion of the index from 700 to I860 documents caused a 

marked slowdown in on-line computer searching time when four or mo-’e 

searchers were on the consoles at the same time, -Sample searches w^re 

run to determine the approximate time required for representative searches. 

Search time ranges for these searches were: 

4 consoles used for FOCUS 

at the same time: - 2-.5 - 6 minutes 

6 consoles used for FOCUS 

at the same time: 6 -13 minutes 

8 consoles used for FOCUS- 5 -IS minutes 

at the same time: 

The slow response time with four or more students at consoles 
led to the following developments that wil 1 - be- briefly described: the 

inclusion of a printed index and an on-line search of a specified range 
of document numbers, 

% 

Computer printed coordinate index- . The- printed index- was prepared 
because of the desirability of having a portable index that could be used 
at any time, as well as because of the- slow machine response time. It 
consists of an alphabetically arranged list of descriptors without cross 
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references. 
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Each descriptor unit containsa list- of descriptor numbers 
representing documents indexed by the descriptor. The document 
numbers are arranged by terminal digit and in ascending order, A sample 
page of the index is given in Figure 3, 

The printed index has been used for the following tasks, some 
of which are described more fully in other sections of the report: 

1, Class demonstration of coordinate index searching; 

2, Aid in selection of termpaper topics and bibliographies; and 

3, Test searching of index (used in combination with on-line 
searched index). 

Search of range of document numbers . The searcher may use either 
the entire data base or, by typing the lower and upper- limit document 
numbers, use only a specified range within Tt, Searches on limited 
data bases require considerably less response time. When demonstrations 
of the system are given to large classes, each console is prepared with 
a set of questions and a data base limited for those questions. Should 
a person want to search the entire index. It is a simple two-step keying- 
in operation to gain access to the whole FOCUS data base- of 1850 documents. 





Figure 2, Instructions- fsr on-line- searciung ruuub 



INSTRUCTIONS for On-Lino Searching: 



1 . 



2 , 

3. 

4* 



5. 

6 , 



Get attention of the computer- Depress ALT CODE key, and while holding xt 
down, press the INDEX key. When you see the cursor that marks typing line - 
Tyne your identification code* )_ : 

Enter it into the computers Depress the RETURN key. When cursor reappears 

Type either one of the followings . 

a. For a display of bibliographic citations, types )LOAU 3 

b. For a display of document numbers, only, types )LOAD 4 

Enter your typing by pressing the RETURN key. When cursor reappears 
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8 , 



9. 



To let the computer know you want to enter a logical search statement, 

Press RETURN key to^ enter it. When the message "ENTER LOGICAL DESCRIPTION" 

■ appears on the screen — . A . . 

Type your logical search statement which should consist of: descriptor 

numbers connected with "AND" "OR" "NOT" (parentheses, if needed) 

After proofreading your typing, enter it by pressing RETURN key. 



10 . 



11 . 



If, in step #4 you chose )L 0 AD 3 , the computer will indicate the number of 
documents that are potentially relevant to your search statement. Xou now 
must use the light pen to indicate your choice of: 
a, viewing the first document citation,' or 

b« returning to step 1 !& in order to type another search statement, 

(If you choose to view a document citation, you will use the light pen, to 
make a choice at the end of each citation. Press the symbol before the 
word you choose. The point of the pen contacts a coded area.) 

Xf\ Xn step $4 you chose )L0AD 4 , "the computer will indicate the number of 
documents that are potentially relevant to your search statement and ask you 
to indicate with the light pen your choice of viewing or not viewing the 
numbers of these documents. 

When you do not wish to continue entering logical statements, 
to get off the computer types QUIT 
Enter it by pressing the RETURN key. When the cursor reappears — 

■ Types )0FF ' 

TO ERASE ERRORS MADE IN TXPINGs (This can be done only before pressing the RETURN key.; 



12 . 



13. 

14. 

15 . 



A. 



B. 



To eras# one character at a times Press the ALT CODE key, and while holding it down, 
press the BACK SPACE key as many times as needed. - 

T© erase an entire lines Press the ALT CODE key and the upshift of the • 



? ? ? OTHER PROBLEMS ? ? ? 
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ALT] 

CODE 



TA8 



3 . 

4 . 
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Did you raaanber to use the special APL characters (rather than the Coursevriter ones) 
when you were typing? 

2, Did you proofread your typing before pressing 

the RETURN key to enter it into the computer? 

Perhaps you confused 1 and 1, or 0 and 0? 

Did you remember to enter your typing by pressing 
the RETURN key? 

Did you get an error message? Call for help, 
when needed. 

Is nothing happening? 

a. Perhaps the computer needs time for a 

lengthy search*— or to service another 
teralnal or teminals. 

Perhaps something is wrong with the system 
or your.teminal. Are others able to 
use their terminals? 
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b. 
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DEVELOPMENT OF TEACHING MATERIALS 

Computer-aided instruction , The original index searching 
exercise written in Coursewriter II and described- in- Tech Memo No. 22 
has been completely revised, The new program, called SEARCH, includes 
six questions and retains the- basic elements of the index searching 
operation, as stated in the decision- making-model , As- in the previously 
used exercises; the correct answer in all but the first step is based 
on the correct answer given to the student for the previous step. The 
six questions (Appendix A) are on a printed handout which is used at 
the console. Sample CRT displays from SEARCH are given in Appendix B. 

A major change in the Coursewriter program is the addition of 
immediate feedback to the student in the form of brief explanations of 
the correct answer to each question or step. This response appears regard- 
less of the student's answer. Another change is the simulation of the 
actual searching of the index in the Coursewriter program instead of a 
searching of the index in the APL language, Until this change was 
made, the entire computer system had to be halted- to- change machine 
language. This was done only once during each of the students' two 
console sessions and every student had to complete the Coursewriter II 
sections (steps one through four as listed on the following pages) for 
each question before the machine change to the APL language could be made 
for step five. The machine language change took about five minutes. By 
eliminating the actual entry of the logical statement into the index 
for on-line searching and by instead simulating the- index response, i.e, 
the display of potentially relevant document numbers, the student is able 
to work through all of the steps for each question- in the correct order 
and progress at his own speed. -Thus, in the revised version of. the 
exercise, the student is given the document numbers resulting from the 
search rather than searching the Index itself, 
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The initial step of the- index searching- exercise*- question analysis, 
attempts to establish the- student's understanding- of- the question, He 
considers each of five fictitious document titles- as- it- appears on the CRT 

In relation to the question which he has in printed- form* A three-level 
(relevant, possibly relevant ,-non^rel event) -relevance judgment is made for 
each title. Following the student's relevance judgment, the correct 
answer for that document title with a brief explanation appears on the 



screen. 

In step two, the student selects indexable information from the 
question. The printed form of the question has most words or phrases 
labelled with numbers in parentheses (Appendix A), The console screen 
displays each word or phrase in its original sequence and the student 
decides whether or not the term represents indexable information by 
selecting either the- "yes 1 - or "no" answer. The correct- answer appears 
with an explanation when the next word or phrase is shown. 

In step three, translation- into descriptors,-a divided screen 
initially displays as possible translation choices instructions, the 
indexable term to be translated, and several descriptor code numbers, 
The student uses the Sal for the vocabulary translation which should 
be to the one or more descriptors whose code numbers- are shown on the 
screen. He then types the selected code numbers and, in response, is 
shown the descriptor in natural language for each of- the pre-selected 
code numbers. Also given is a brief statement for each as to whether 
or not it is correct and why, 

A revised format for step four, search logic formulation, was 
developed, The student is shown each descriptor and code number to be 
used, as well as a statement form of only the code numbers connected 
with blank spaces. The student types the necessary connectors ("and", 
q "or", "not") and parentheses in- these blanks to create the logical 

ERIC ' ' * 18 



statement. The correct statement Is displayed -after the student's 
response. This format proved to have mechanical limitations both 
in user operation and printout analysis, In view of this, the ^free- 
form" response used - in the original index searching Coursewriter II 
program when the student typed- in the entire logical statement seems 
preferable. 

In step five, the selection of relevant documents, the console 
screen displays the document numbers obtained for the question from a 
previous search of the index. The number of documents ranges from four 
to nineteen for the six questions. The student reads the document abstract 
in the Abstract Bulletin and indicates his three-level relevance judgment 
at the console. This format provides a summary and- an Hern analysis of 
the relevance decisions as part of the computer printout , not, as in the 
original index searching exercise, on separate printed forms manually 
recorded by the student, 

In early 1971, PREP, an index preparation exercise using computer- 
assisted instruction and comparable to SEARCH, was developed. PREP is 
also based on the five decision making steps in the-model , is written 
in Coursewriter II, and contains six abstracts with- bibl iographic citations 
which are reproduced on printed handouts used at the- consoles, A sample 
abstract (printed handout) and sample- CRT displays for PREP are given 
in Appendix C, Step one, the selection of documents for inclusion in 
the index, is presented on the screen as a series of eleven fictitious 
and very brief document abstracts, (These abstracts are different from 
the six used for steps two through six of the exercise,) For each of 
these abstracts the student decides whether or not it should be included 
in FOCUS and then he is shown the correct answer with an explanation. Step 
two, the selection of indexable information, has the same format as the 
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comparable step in the index- searching- exercise,- -The- abstracts, in printed 
form, have numbers in parentheses following most- words or phrases. The 
numbered terms per abstract vary from- ten to sixteen,- These terms are 
repeated on the console screen and the- student decides- whether or not 
the term is useful as indexable information by-making- a- "yes" or "no” 
choice for each, A brief explanation accompanies the display of the 
correct answer. As a final -part of step two, the student characterizes 
the abstract as one of the six types of documents used in the index or 
indicates that this type of descriptor does not apply. 

The divided- screen format for step three,- the translation of 
indexable information into descriptors, is the same as that used for this 
step in SEARCH, For the six abstracts the number of terms to be translated 
ranges from six to ten per- abstract. Translation- is restricted to Sal 
descriptors which are exact matches (singular and plural forms of a 
word are considered exact matches) or stated equivalents of the terms, The 
console screen displays code numbers of descriptors -which are possible 
translation choices (usually three) plus the additional choices of "Do 
not use" and "New Sal decision," After the student types in his answer 
of a number (s) or phrase, the full screen display- identifies the descriptors 
of the code numbers and gives the- correct "yes" or "no" answer with 
an explanation for each. The "New Sal decision" is the correct choice 
when the index vocabulary does not currently contain the indexable term 
as a descriptor (exact match) or a stated equivalent- (see reference). 





In step four there- are- four choices of- a- new- Sal decision 
for a term: 

1, Omit the concept entirely-as not being important', 

2, Make a "see" reference to-a single existing descriptor! 

3, Make a :! see :i reference to two or more existing descriptors' or 

4, Create a new descriptor. 

The terms presented in step four are those which required a "New 
Sal decision" In step three, translation into descriptors. The decision 
choice is based on the FOCUS Sal indexing rules with which the student 
is already familiar. After the student indicates- his decision, the 
correct responses for each of the four choices with short explanations 
appear on the screen. If the correct answer is a ^see" reference 
(choice 2 or 3 above), the next screen display- shows- the tem fcftd <1 
statement of the type of Sal decision that is to-be made. In a few 
instances the student is given the- descriptor (s) to which a ’‘see" 
reference is to be made, but usually he is presented a divided screen 
showing code numbers (the step-three, translation, format). Using 
the Sal, he selects- the- descriptor(s) to be used- in- the- cross-reference. 
The descriptors and correct responses with explanations for each code 
number appear after the student- answers . If the new Sal decision is to 
create a new descriptor (choice 4 above), the student selects the correct 
BT (broader term). 

Step five, updating the Sal is based, as were steps 2, 3, and 4 
on the correct decision in the previous step, The screen format is essen- 
tially "free-form. " For cross-references, the student -is asked to type 
on the first frame the proper see- reference and on- the second frame the 
proper see from reference(s), For new descriptors the- desired response is 
the proper descriptor- entry followed by the NT (narrower term) entry under 



the BT (broader term),- The correct form of the- Sal - updating entry 
Is displayed after the student enters his answer) 

Programmed text . In early- 1971-,- a- programmed- text- was developed 
on index preparation to- be used- by- the students' in lieu- of- the computer- 
assisted instruction -based PREP- program. -Two limitations of the CAI 
program led to this decision The -first- is the-llmited number of 
characters that can be displayed on the CRT at one time. There are 
sixteen lines on a screen, each with a maximum -of forty characters, In 
practice, all sixteen lines of a screen cannot be used because of spacing 
considerations for readability and the need to save lines for student 
responses. This proved to be a problem, particularly in step five 
where a descriptor with a broad term and narrow terms- and/or cross- 
references had to be displayed. The second limitation with multiple line 
responses also occur -ed in the same step. Students typically needed to 
enter two or more lines of answers for step five,- This required different 
machine instructions for the first and second lines- of input and caused 

the students difficulties. 

The basic volume* ran P rt Co ordinate Index - Prepara 
Test (the CIP Text), uses branching throughout It- serves solely as a 
teaching device, requiring- no- written answers and nothing- to- be handed in 
by the student. Three separately- bound exercises provide the student 
with practice in indexing three abstracts using the- techniques learned with 
the programmed text. The CIP Text is based on rules for FOCUS and uses 
examples from the FOCUS Sal, For this reason, the student- needs a copy 
of the FOCUS Sal when reading the text. Sample pages from the CIP Text 
are given as Appendix D, 

The programmed text is based on the five decision-making steps 
in index preparation. An explanation is given with each correct answer 
for all exercises and examples, and clues or explanations are given 
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with all incorrect answers which lead the student- to the correct 
answer, 

Step one, the selection of documents for- inclusion in the 
index, is explained and the FOCUS- rules- for selection are- stated. Two 
examples are discussed before the student is presented- with five documents 
as exercises in document selection, 

A detailed presentation of the FOCUS rules introduces step two, 
the selection of indexable information. One document abstract, with most 
terms or phrases numbered, is given as an example for- selecting index 
terms. The student then has two additional document abstracts, one 
with fourteen and one with fifteen terms, to use as exercises. Following 
each term is the correct answer, a brief explanation, and the FOCUS rule 
number upon which it is based. Each document also has a "type of document 
descriptor choice. 

The discussion of step three, translation of indexable information 
into descriptors, begins with the creation of a prototype or mini-Sal from 
a list of about twelve related terms. First the concepts are arranged 
hierarchically , then the Sal entries are established. One list serves as 
an example and the student is asked- to create mini -Sals- for- two additional 
lists of terms. The correct hierarchies and Sal entries are given for 
a 1 ! terms. 

The FOCUS Sal rules are- presented with examples- of each, Then, 
the student has five indexable terms to translate into descriptors using 
the Sal, Detailed- explanations of the correct answers are given. 

The reasons for the four choices of new Sal decisions for step 
four are presented, with specific examples of each. The student is given 
two terms as an exercise in choosing the correct new Sal decision. 
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The three types of Entries- for updating the Sal , step five, are 
illustrated with entries front the Sal The- student- Is given three 
exercises, each of which has four- possible -Sal entries for the 
answer. 

There- are three separately bound document abstract exercises which 
supplement the CIP Text-. Each- has a separate- printed- answer sheet on 
which the student records his- initial responses. - The exercises carry the 
documents through all five decision making steps in Indexing. When an 
incorrect answer is chosen, a clue is given as to what is incorrect and/or 
what the right answer is. Explanations are given for all correct answers. 
References are made to rules and discussions in the- CIP Text and to 
entries in the FOCUS Sal, The exercises are branched- and the student 
must select the correct answer to- any one segment to proceed. 
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THE USE OF FOCUS IN TEACHING 



Demonstration of an on-line searched coord-tnate index to 
students in the introductory information science course . During the 
past year and a half, the index was used by students in LIS 586, Infor- 
mation Science and Libraries, as ''hands on experience with both an on- 
line searched computer and a coordinate index. LIS 586 is required of 
all master's students in the library school and is offered every term. 
FOCUS was demonstrated by means of half hour sessions at the console. 
During these sessions, students are asked to perform simple coordinate 
Index searches already formulated for them. After the sample questions 
are completed and if time oermits, students can search any other questions 
of interest to them. No formal evaluation was made of the sessions at the 
console but a number of students commented that the experience v/as useful 



in reinforcing material covered in the lectures and the readings. 

Preparation of student papers with and without the aid of FUCUS . 
While the assignment of essays or papers based on a review of the litera- 
ture is commonly used in a number of courses, including courses in 
library schools, relatively little is known about how students select 
topics for papers (when they don't have a file of previous papers to 
draw on) and how much time such exercises take. A small scale study was 
conducted, in part to answer the above questions and in part to determine 
whether FOCUS can be used as an aid for the preparation of papers. A 
class of thirty two students enrolled in LIS 586 was asked to prepare 
short papers on any one of the topics discussed in the course, topics 
that are also included in FOCUS, The papers were submitted in two 
parts. Part one, the part which provided data for this study, consisted 
of the title and outline of the paper, a short bibliography of at least 
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Six items, a description of the methodology used- in selecting the topic, 
preparing the outline and the bibliography, a record of the time taken 
for this task, and comments- on- the methodology used v - Part two was the 
completed paper,- -The- cl ass was divided- -into two-groups, The groups 
which were also used-in- the- study- of- different- teaching methods 
mentioned later in this section, were equal in number- and comparable in 
terms of library school grade point averages and scores on the Graduate 
Record Examination, One group was instructed to use only FOCUS for 
the selection of the topic, preparation of the outline and bibliography, 

The other group was instructed to use ar, .;iher too- or combination .- 
tools except FOCUS. 

The student prepared record of the methodology was not as 
complete as we had hoped to obtain and the sample is probably too 

small to permit any sweeping, generalizations,.. Nevertheless, the following 

observations are reported; 

Students in both groups- typical !y selected two-concept topics 
with the concepts being combined as logi'-a'’ products, e-,g, computer- 
for selective dissemination of information-, fas-: 1m lie transmission a no 
library networks! teletypewriters for interlibrary loans. The time 
taken for this task varied widely within the groups even- though roughly 
similar products were turned in. -The time range for the FOCUS group 
was 2 to 19 hours with an average of 8.4 hours. The- time range for the 
non-FOCUS groups was 1 to 17 hours with an average of 7.8 hours. There 
is no statistical difference between these two groups at the 0,10 level 

as per Mann-Whitney U test, (Siegel, 1 956 ) ^ 

Over half of the FOCUS group had difficulties- in using the 
index for this purpose and/or expressed- dissatisfaction- at being restricted 
to a single tool. Three-fourths of the non-FOCUS group used two or more 
of the following tools; Library Literature , Library and Information Scienc e 
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Abstracts, Information Science Abstracts, and Research- in 'Education, 

* ------ t ^ - I ,% w m * IL y m *m - ■ ■ 

One-fourth of the non-FOCUS group used Library Literature only. 
Difficulties in selecting a topic were encountered by both groups. 

These included insufficient acquaintance- with- the- field- for querying 
the index, redundancy of the 1 iteratu^e, and unavailabil ity in the 
library of the selected references. Results from this study suggest that 
FOCUS, or any other single tool for that matter, should not be used 
for such an exercise and that a more extensive study of the rationale 
for assigning papers and method of preparation and evaluation might be 
useful . 

Comparison of lecture and multi-media methods for teaching index 
searching . In LIS 586, indexes are discussed from a user's (searcher) 
rather than a producer's (indexer) point of view. This topic takes up 
about one third of the course. With the exception of the aforementioned 
demonstration of FOCUS and an index searching exercise, the subject is 
presented primarily via the lecture method. During the winter and spring 
terms of 1 970/71 , a portion of LIS 586 students were exposed to indexes 
via multimedia rather than the lecture method. Miss Drucl 11a Motley, 
one of the doctoral students of the School of Library- Science, has prepared 
as part of her dissertation tape-sl ide- presentations programmed texts, 
and computer assisted instruction for presenting- this- material . The 
difference between the lecture and multimedia teaching methods in terms 
of pre-test/post-test scores, attitude- changes toward subject, and time 
taken by students is now being- analyzed,. The results of this study will 
be reported in Miss Motley's dissertation. 

The use of FuCUS in the abstracting and indexing course . The 
objective of the indexing section of LIS 587, the five quarter hours 
abstracting and indexing course, is to acquaint students with the 
O 
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techniques as well as the problems- of- index preparation- and evaluation 

This was done during the seven- weeks segment of- the- course devoted to 

indexing by means oi the following steps; 

1. Instruction in indexing 

2, Student indexing 



3, Planning of index evaluation 




4, Carrying out index evaluation 

5, Discussion of results of index evaluation 

The eleven students enrolled in this course in the Spring term 
of 1971 had been -introduced to FOCUS searching as students in LIS 586. 

Index preparation was taught by means of the previously mentioned programmed 
text. Students were given a pre-test prior to reading the programmed text 
and doing the associated exercises, A post-test was- given at- the completion 
of the reading of the programmed- text and the indexing exercises. The 
pretest/ posttest is reproduced as Appendix E. - The- pre-test scores range 
from 60% to 76% and- average- 68%,- The pastiest- scores- range from 68% to 

98% and average 87%. No comparable data is available- for teaching index 
preparation with the aid of the previously used classroom instruction 
method since no equivalent tests were used- with this technique. The 
relatively high scores in the pre-test are attributable to the students' 
exposure to FOCUS searching in LIS 586 and to the similarity between 
the steps in index searching and preparation. At- the completion of 
the posttest, students indexed three common document- abstracts in class. 
Then each student was given- five- different abstracts to index. Index 
entries for the 55 document abstracts so obtained- were- added to the 
on-line searched index.- new indexing decisions were included in a 
supplement to the Sal, and a supplement to the- abstract bulletin was 
prepared. The indexing was followed by a class discussion of index 
evaluation and a plan- for conducting- an index evaluation. The class 
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decided that each student should formulate one question based on a 
document indexed by another student (a single source document question) 
and four questions based on searching the entire document collection, 
now over 1900 documents. The latter type of questions is to-yield between 
two and fifteen documents. No restrictions were placed- on allowable 
search strategy. The search results in terms of relevant and non-relevant 
documents and the search strategy used for each question were recorded 
on a form designed by the students. The completed forms were used to 
calculate recall and precision ratios as well as for analysis of reasons 
for search failure. Relevance was judged on a two level basis (relevant 
and non-relevant) from reading the document abstract. The searcher 




was not permitted to negotiate the question with the search formulator 
since the students felt that such negotiation might result in a giving away 
of the answers. Eleven single source document questions and twenty one 
multiple document questions were searched by the students. A number of 
questions were eliminated because they were either duplicate of or 
closely related to questions on the list. One question was eliminated 
because the search formulator used- incorrect search- logic in answering 
his own question. Each student searched one single source document 
question and at least five other questions assigned to him. Several 
students used the printed index for both preliminary question formulation 
and searching. The complexity of the search logic made the on-line 
mode more suitable for later search stages. Search results recorded on 
the previously mentioned form were analyzed by the search formulator. His 
first task was to compare relevant documents that he selected with those 
selected by the individual searchers. If additional relevant documents 
were located by the searchers, the search formulator decided whether or 
not to accept these documents as relevant, Acceptance of additional 
relevant documents required an adjustment of searcf^ results for calculation 
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of recall and precision ratios as well as for characterizing reasons for 
non-retrieval of relevant documents, The most common reasons for non- 
retrieval of relevant documents- listed- in- decreasing- order- of occu^ance 
were: difference in relevance- judgment- between search formulator 

and searcher, inadequate translation- of indexable information in the 
question into descriptors, inadequate search- strategy- - search too 
specific, inadequate search strategy - search too generic, misunderstanding 
of question, and clerical error. 

In lieu of a final examination- for this portion of the course, 
students were asked to prepare a case history of the index evaluation 
exercise and to comment on- this experience. The- case histories indicate 
an understanding of the problems In Index preparation, search* ng, < no 
evaluation. Several suggestions- aimed- at improving the index evoiuct ! cn 
procedure were made. These Include suggested changes in- the search record 
form, the recording and subsequent analysis of search strategies that 
yield no relevant documents-, and the use- of questions posed by students 
in another class to make the test more realistic. 

The objectives of the indexing- section of this- course are to 
acquaint students with index preparation and evaluation, -These are broadly 
stated objectives whose- achievements are difficult to- measure. We cannot 
present quantitative measures of- student performance,- However, it is our 
Impression from reading the case histories turned- -in- by- the students 
and watching their performance that- the index evaluation exercise presented 
the students with a challenge, - They responded- well- to this challenge by 
working at least as hard as was expected of them and by- coming up with 
conclusions about index preparation and evaluation that were not novel 
but showed an insight into the problems involved. 
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• FUTURE FLANS 

We plan to continue the- use* of FOCUS to demonstrate on-line 
searches of coordinate -indexes- -ift Llie- introductory -information science 
course and for Index preparation,' searching-*- -and- evaluation' in the abstracting 
and indexing course. It is-our hope tnat the program- can- be run on the 
University's CDC 6400 Computer which now has on-line searching facilities. 

The principle advantage of using the CDC 6400 would be the greatly reduced 
search time on this more powerful computer. 

The indexing system will also be used as a testbed for determining 
the effect of changes in different components of the index on index 
performance. Variables to be tested include question characteristics, index 
vocabulary, and search output. Also planned is the- use of the techniques 
and programs developed in the course of this study for the preparation of 
indexes to other data- bases- as for example- an- inventory- of skills of the 
Florida State University- faculty. We would like to prepare both a 
printed and an on-lind searched "Who Knows What" directory. This is 
to serve as an aid in connecting individuals with a need for information 
with individuals that have the needed information,- Case Histories of 
the use of the system are to be collected both- to- study- system performance 
and this aspect of information gathering habits- of members* of an academic 
community. 
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FOOTNOTES 



1 The work for this project- was made- possibl e- through- funds from the 
Office of Naval Research, Project NR 154 - 280 *' - We- are- grateful for this, 
support , The authors also- wish- to- thank- Tom- McMurchie- for- his programming 
work. 
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APPENDIX A 

QUESTIONS USED FUR THE CAI COURSEWRITER II 
INDEX SEARCHING EXERCISE (SEARCH) 

1. Does (1) a (2) public library (3) in California (4) use (5) 
phototypesetting machines (6)? 

2. I am looking (1) for a public library (2) which has all or 
part (3) of its bindery operations (4; computerized (5), 

3. Have keyword from title indexes (11 been used (2) in biology 
(3)? Fields (4) closely related to biology will be 
acceptable (5), 

4. Is there anything (11 written on centralized (2) reference 
referral networks (3) dealing (4) specifically (51 with 
maps (6) and charts (7)? 

5. Are there any (1) discussions (.2) of reasons (3) for failures 
(41 or (5) false drops (6) in machine searches (7)? 

6. Are (1) any school libraries (21 using (3) computer terminals (41 
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APPENDIX B 



SAMPLE CKT SCREEN DISPLAYS FROM SEARCH, A CAI PROGRAM 

QUESTION ANALYSIS: Select documents for 

Question 2, Document title: 

University of Sussex automated house- 
keeping byproducts included request 
forms, accession lists and bindery 
records, 

□ Relevant □ Non-relevant □ Possibly rel 
Non-relevant; not a public library, 

□ Continue 



SELECT INDEXABLE INFORMATION: Question 2 

1, looking i 

□Yes □ No j 

No-carries no meaning j 

2. public library j 

□Yes □ No j 

Yes-organization 

3, all or part 

□ Yes □ No 

No-can 't search this specifically 

4. bindery operations I 

□ Yes □ No , I 



Yes-information service or operation 
□ Continue 



TRANSLATE WITH SAL: Question 2 (Type 
number of each descriptor to be used) 
public library 

Publ i c 1 i - 508-Yes-match 
brari es 

Regional li- 539-Yes-SA of Public li- 
braries brari es; related 

State 1 ibraries612-Yes-SA of Public li- 
braries; related 

□ Continue 
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Sample CRT screen displays from SEARCH - continued 



SEARCH LOGIC: Question 2 
Type "and" "or" "not" "(" ")" in blanks 
57 Binding; 508 Public libraries; 539 
Regional libraries; 612 State libraries 




—57 508 539 612— 




57 and (508 or 539 or 612) 




□ Continue 




DOCUMENT SELECTION: Question 2 
Characterize relevance of each document 
using Abstract Bulletin, 

Document 531 U Rel □ Non-rel □ Possibly rel 
Possibly relevant; "aims" doesn't clarify 
if it's operational or only planned for. 




Document 844 □ Rel □ Non-rel □ Possibly rel 
Non-relevant; binding of computer- 
produced catalog. 




Document 1273 □ Rel □ Non-rel □ Possibly rel 
Possibly relevant; machine portion of 
system may include binding. 




□ Continue 
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APPENDIX C 

SAMPLE ABSTRACT AND CRT DISPLAYS FROM PREP 
A CAI COURSEWRITER II EXERCISE 
IN INDEX PREPARATION 

Sample abstract 
index preparation 



Magnavox (l) vs, LDX (2) in interlibrary loans (3), Vincent 
Burgess. ALA Bull, 35, 727-729, Nov 69 (4). 211 lus 

1 table. 



An (5) experiment (6) in using (7) the Magnavox tele- 
copier ( 1 ) and Xerox LDX facsimile transmission (2) for 
inter library loans (3) of journals (8), The (9) expense (10 J 
and rapidity (ll) of the equipment (12) were compared (13) 

The results (14) were reported (15) at a 1968 (16) meeting. 



Sample CRT screen displays 



SELECT INDEXABLE INFORMATION: Abstract 1 

1. Magnavox telecopier 

5 Yes □ No 

Yes-name of equipment 

2. Xerox LDX facsimile transmission 

9 Yes _ □ No 

Yes-name of equipment 

3* interlibrary loans 

9 Ye ? _ . . . □ No 

yes- 1 ntormation service- or operation 

4, 69 

□ Yes q No 

Yes-date of publication 

5. An 

9 ^ es . □ No 

No-carries no meaning 

□ Continue 



.. * f t !^ ent answers are not shown, but CRT responses 
that follow student answers are shown here. 



correct answers) 
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SELECT INDEXABLE INFORMATION: Abstract 1 

type of document 
Check the correct answer: 

Bibliography □ 

Case histories □ 

Philosophy □ 

Research □ 

Reviews □ 

Surveys □ 

Does not apply □ 

Research - an experiment is research 



□ Continue 



TRANSLATE WITH SAL: Abstract 2 (Type des 

criptor number (s) to-be used or phrase 
Xerox LDX facsimile transmission 
Data transmis- 165-No-too broad 
sion equipment 

Telefacsimile 646-Yes-see reference 




Copiers, 201-No-see=ref from Xerox 

photocopying 914 not Xerox LDX 

Do not use -No-equipment is in- 

dexed 

New sal decision -No-see reference ex- 
ists in sal 




□ Continue 




Abstract 2 

New sal decision: Magnavox telecopier 
A, Choice of decisions (check only one) 




1, Omit concept □ No-types of equip- 
ment are indexed 

2, See ref to single existing descrip- 
tor □ Yes-specific descriptor for tele- 
copiers is available 

3, See ref to two or more- existing des- 
criptors □ No-single descriptor is 
adequate 




4. New descriptor □ No-existing descriptor 
is adequate 




□ Continue 


% * ^ 
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New sa! decision: Magnavox telecopier 

Magnavox telecopier will be-added to the 
sal by expanding- the existing descriptor 
"Telefacsimile-' with a cross reference, 

□ Continue 



New sal decision: Magnavox telecopier 
B, Update sal: type "see : reference's) 
to existing descriptor (s) , Don't type 
BT's, spope notes, NT's, X's, SA's 
that are already printed in the sal. 



Magnavox telecopier see Telefac- 
simile 646 

□ Continue 



New sal decision: Magnavox telecopier 
B, Update sal : type "see from" refer- 
ence^! of existing descriptor(s) . 

Don't type BT's, scope notes, NT's, X's, 
SA's that are already printed in the sal 



Telefacsimile 646 

X Magnavox telecopier 



□ Continue 



APPENDIX D 



SAMPLE PAGES FROM COORDINATE- INDEX- PREPARATION , 

A PROGRAMMED TEXT 

Programmed Text, bottom of p, 49 

Indexable information term #4: Chained storage 



Programmed Text, p, 50 

The see reference for the process Chained storage is on 
SAL p, 32, It refers to two descriptors connected by "and" 
which, of course, means that the concept "chained storage" must 
be characterized by both of these descriptors each time it is 
indexed or searched in FOCUS, Under the descriptor Storage, of 
words in computer 617, SAL p, 22!, you will find the see from (X) 
reference for Chained storage followed by Distribution in parentheses 
which Indicates that both descriptors are required for- this concept. 

This kind of tracing also provides- the editor with- complete information 
for SAL revision. It leads him to all of the- SAL locations of this 
concept. If he was revising the descriptor Distribution 183, SAL p, 62, 
he could follow- the tracings and make the necessary- changes under 
Chained storage. Storage, of words in-computer, etc, 



Indexable information- term #5: -Science Citation Index 



Programmed Text, top of p, 51 

The see reference for the information- source Science 
Citation Index is on SAL p. 206, It refers to the subject"’ 
descriptor. Science and technology, and to two types of 
publication descriptors. Citation indexes and Information 
sources, machineabl e, incompliance with the SAL rule for in- 
formation sources (p. 44), 

On SAL p, 206 the descriptor Science and technology 572 
has the X reference for Science Citation Index , In paren- 
theses following It are the two' additional descri ptors that 
are required to represent this concept in FOCUS-, By looking 
up both of these terms in the SAL (pp, 106-107 for Information 
sources, machineabl e, and p, 35 for Citation indexes) you can 
see the full tracings for this concept. 




31 

4 i 



32 



Programmed Text, p, 70 

In this exercise you will select the letter of the 
correct SAL updating for each of three terms, (Be sure 
the answer you select Is complete . ) 

SAL Updating exercise #1:- term: Layout 

Index- language equivalent Formats 

.... ... 223 

- • - select a see- reference to update 
the SAL 

A, Layout 

BT Properties, other 

Properties, other 
NT Layout 
NT Formats 



B. 


Formats 


223 


see Layout 




Layout 


see 


Formats 223 


C. 


Formats 


223 






X Layout 






Layout 


see 


Formats 223 


D. 


Formats 


223 





SA Layout 



Layout see Formats 223 



If you select answer A go to p, 71 
If you select answer B go to p, 72 
If you select answer C go to p. 73 
If you select answer D go to p, 74 




42 



33 



Programmed Text, p, 71 

If you chose answer A for Layout you will probably 
want to review p. 67 on see references , Remember that BT 
and NT establish a place for the term- in- the hierarchy 
which means the term becomes -a descriptor, 

After you review p, 67 return to p,- 70- to try again. 



Programmed Text, p. 72 

If you chose answer B for Layout you will probably 
want to review p. 67 on see references. Remember the see 
reference must have a see from reference going in the 
other direction. 

After you review p, 67 return to p, 70 to try again. 



Programmed Text, p, 73 
GOOD. 

Answer C is correct Tor-Layout, Layout is the equiva- 
lent to Formats so only one term needs to be used as a des- 
criptor. Since Formats is an established descriptor, the 
see from (X) reference is added to it and a see reference 
is added to the SAL. 

Continue to p, 75, 



Programmed Text, p, 74 

If you chose answer D for Layout you will probably 
want to review p, 67 on see references. Remember that the 
see reference must have a- see from reference going in the 
other direction. 

After you review p, 67, return to p, 70 to try again. 
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APPENDIX E 

CuORDlNATE INDEX PREPARATION - Test 

Please record your answers on the separate answer 
sheet provided. Answer each question In the established 
order. 

Document Abstract 

Tests (1) of Uniroyal Microfilm Retrieval System (2), USA 
Libr, Bull., 84, 42-43, May 63 (3), 2 illus. 

Results (4) of the ALA Library Technology Program (5) 
tests (1) of the Uniroyal Microfilm Retrieval System (2) are 
presented. The miniaturization ratio 16) of the microfilm (7) 
exceeds (8) any other microform (9) available In 1967 (10). 
Durability (11) of the cassettes (12) was questioned (13) by 
the Phillips Petroleum Company Library- (14) which has used 
the system (2) one year (15) Inconjunction with a larger In- 
formation storage and retrieval system (16) using the Honey- 
well 1200 computer (17). The entire system (16) is written 
in JOVIAL programming language (18). 



Step 1 Is Selection of Documents for inclusion In the Index, 

You need to decide if this document abstract should 
be included in FOCUS, considering the subject matter* date of 
publication and language. Circle your answer on the answer 
sheet. 



Step 2 is the Selection of Indexable Information, 

Listed below are each of the numbered- terms in the 
abstract. Decide for each term if it should be selected as 
indexable information. Circle your answer for each on the 
answer sheet. 

1 , tests 

2, Uniroyal Microfilm Retrieval System 
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4. results 

5. ALA Library Technology Program 

6. miniaturization ratio 

7. microfilm 

8 . exceeds 

9. microform 
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- Test 2 

Step 2. Selection of Indexable Information - continued 

10, 1967 

11, durability 

12, cassettes 

13, • questioned 

14, Phillips Petroleum-Company Library 

15, one year 

16, information storage and -retrieval system 

17, Honeywell 1200 computer 

18, JOVIAL programming language 



Step 3 is Translation into Descriptors, 

You are given each term of indexable information separately 
accompanied by a list of descriptor-' code numbers and phrases, The 
numbers are the actual descriptor codes given in the SAL (Subject 
Authority List). Two different phrases are usually included m the 
answer list. One phrase, "Do not use, ;i should be selected when 
the indexable term has already been used (indexed) for- that document 
or when a narrower (more specific)- level -descriptor has- already been used 
(indexed) for the document. -The second- phrase, -"New- SAL -decision," should 
be selected when you- eannot- translate- the- indexable term into 
descriptors through the SAL- and a- new index vocabulary decision is 
required* 

You should select only- the SAL descriptors- that are exact 
matches or stated equivalents (see references)- of - the units of 
indexable information. The plural form- of- a- word- is considered 
an exact match. 

Example: Catalog- can^ be translated into the descriptor 

"Catalogs- 93" because' the plural word form 
is considered an exact match, 
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Test 3 

Step 3. Translation into Descriptors - continued 

Example: General Motors Library^- cannot be translated 

into the descriptor "Industrial libraries 
281 ” because- the -foil owing --cross references 
(stated- equivalents) are not- in the SAL on 
pp, 82-83 and 101-102: 

General Motors Library -see Industrial li- 
braries 281 

Industrial libraries 281 

» i i 
» • t 

X General Motors Library 



To do the translation step, look up the indexable term 
in the SAL and try to translate it into the index language. 

The answer you decide on should be included in the accompany- 
ing list of numbers and phrases.- Circle your answer (s) for 
each term, J 



Indexable information !, Tests 
translation choices: 

623 

651 

206 

Do not use 
New SAL decision 

Indexable information 2,- Uniroyal Microfilm- Retrieval System 
translation choices: 

400 

731 

403 

Do not use 
New SAL decision 

Indexable information 3, 68 

translation choices: 

716 

900 

902 

913 

Do not use 
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Test 4 

Step 3, Translation into Descriptors - continued 

Indexable information 4, ALA Library- Technology Program 
translation choices: 

340 

28 

Do not use 
New SAL decision 

Indexable information 5, -miniaturization ratio 
translation choices: 

522 

530 

820 

Do not use 
New SAL decision 



Indexable Information 6, microfilm 
translation choices: 

398 

784 

400 

Do not use 
New SAt decision 

Indexable information 7, microform 
translation choices: 

404 

400 

398 

Do not use 
New SAL decision 

Indexable information 8 ■. durability 
translation choices; 

351 

707 

Do not use 
New SAL decision 
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Test 5 



Step 3. Translation' into Descriptors - continued 



Indexable information 9, cassettes 
translation choices: 



402 

400 

89 

Do not use 
New SAL decision 

Indexable information 10* Phillips Petroleum Company Library 
translation choices: 



284 

281 

794 

282 

New SAL decision 



Indexable information 11, information storage and retrieval 

system 

translation choices: 

295 

911 

577 

Do not use 
New SAL decision 

Indexable information 12. Honeywell 1200 computer 
translation choices: 

252 

852 

204 

Do not use 
New SAL decision 

Indexable information 13, JOVIAL programming language 
translation choices: 

495 

119 ' 

328 

Do not use 
New SAL decision 



i 



\ 

i 



j 

I 

l 
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Test 6 

Step 4 Is New SAL Decisions, 

When no descriptor- or stated equivalent- exists in the 
SAL for a unit of indexable information! the indexer has four 
choices for a new SAL decision'. Circle the number of your 
answer for each term on the answer sheet, 



Indexable information 1 , Uniroyal Microfilm Retrieval System 
Step 4. New SAL Decision: 

1. Omit the concept entirely 

2, Make a see reference to a single existing descriptor 

3, Make a see reference to two or more existing des- 
criptors 

4. Create a new descriptor 
Circle the number on the answer sheet. 



Indexable information 2. miniaturization ratio 
Step 4. New SAL decision, 

Is the new SAL decision choice #1 , Omit the concept 
entirely, the correct decision for this term? Circle "Yes" 
or "No“ on the answer sheet. 



Indexable information 3. durability 
Step 4. New SAL Decision 

Is the new SAL decision choice #1 , Omit the concept 
entirely, the correct decision for this- term?- Circle "Yes 11 
or "No" on the answer sheet. 



Indexable information 4, Phillips Petroleum Company Library 
Step 4, New SAL Decision, 

1, Omit the concept entirely 

2, Make a see reference to- a single- existing descriptor 

3, Make a see reference to two or more existing des- 
criptors 

4, Create a new descriptor 
Circle the number on the answer sheet. 
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Test 

Indexable information 5, Honeywell 1200 computer 
Step 4, New SAL Decision. 

1. Omit the concept entirely 

2. Make a see reference to a single existing descriptor 

3. Make a see reference- to twoor more existing des- 
criptors 

4. Create a new descriptor 
Circle the number on the- answer sheet, 



Indexable information 6, JOVIAL programming language 
Step 4, New SAL Decision. 

1. Omit the concept entirely 

2. Make a see reference to a single existing descriptor 

3. Make a see reference to two or more existing des- 
criptors 

4. Create a new descriptor 
Circle the number on the answer sheet. 
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Test 8 

Step 5, Updating the SAL 

After a new indexing decision has been -made, the SAL 
must be updated (revised) to reflect the- change. Circle 
the letter of your answer- for- each term- on- the- answer sheet. 



Indexable information 1,- Uniroyal Microfilm Retrieval System 

Step 5, Updating the SAL 

Descriptor: Microform search systems 403 

Expand an existing descriptor with a see reference. Circle 

the letter on the answer sheet. 



A. Microform search systems 403 

NT Uni royal Microfilm Retrieval System 

Uni royal Microfi lm Retrieval System 
BT Microform search systems 



8, Microform search systems 403 

NT Uniroyal Microfilm Retrieval System 

Uniroyal Microfilm Retrieval System see Microform 
search systems 403 



C, Microform search systems 403 

X Um royal Microfilm- Retrieval System 

Uniroyal Microfilm Retrieval System See Microform 
search systems 403 



D, Uniroyal Microfilm- Retrieval System- - see Microform 
search systems 403 



O 
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Indexable information 2. • miniaturization ratio 
Step 5, Updating the SAL. 

Descriptor: Reduction ratio 820 

Expand the existing descriptor with a- see reference, 

the letter on the answe/ sheet. 



A. Miniaturization ratio 
BT Reduction ratio 

Reduction ratio 820 
NT Miniaturization ratio 



B. Miniaturization ratio see Reduction ratio 820 

Reduction ratio 820 
X Miniaturization ratio 



C, Miniaturization ratio see Reduction ratio 820 

Reduction ratio 820 
SA Miniaturization ratio 



D, Miniaturization ratio 820 
X Reduction ratio 

Reduction ratio see Miniaturization ratio 820 
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Test 9 



Circle 
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Test 10 

Step 5, Updating the SAL. 

Indexable information- 3-. durability 
Descriptor: Wear 707 

Expand the existing descriptor- with a- see reference. Circle 
the letter on the answer sheet, 



A, Durability see Wear 707 

Wear 707 
X Durability 

B. Durability 707 

X Wear 

Wear see Durability 707 

C, Durability 

SA Wear 

Wear 707 
SA Durability 

D. Durability see Wear 707 

Wear see also Durability 
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Test 1 1 



Step 5, Updating the SAL, 



Indexable information- 4. ■ - Phil 1 ips- Petrol eum- Company Library 
Descriptors: Industrial libraries 281 

Petroleum 794 

Expand the existing descriptors with- a- see reference. Circle 
the letter on the- answer sheet. 



A, Industrial libraries 281 

X Phi 111 ps Petrol eum- Company Li brary 

Petroleum 794 

X Phillips Petroleum Company Library 

Phillips Petroleum Company Library see Industrial li- 
braries 281 and Petroleum 794 



B, Industrial libraries 281 

NT Phillips Petroleum Company Library 

Petroleum 794 

NT Phillips Petroleum Company Library 

Phillips Petroleum Company Library see Industrial li- 
braries 281 and Petroleum 794 



C, Industrial libraries 281 

X Phillips Petroleum- Company Library (Petroleum) 

Petroleum 794 

X Phillips Petroleum Company Library 

Phillips Petroleum Company Library see Industrial li- 
braries 281 and Petroleum 794 



D. Industrial libraries 281 

X Phillips Petroleum Company- Library (Petroleum) 

Petroleum 794 

X Phillips Petroleum- Company Library (Industrial libraries) 

Phillips Petroleum Company Library see Industrial li- 
braries 281 and Petroleum 794 
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Test 12 

Indexable information 5, Honeywel 1 1200 computer 
Step 5, Updating the SAL. 

Descriptor: Honeywell 1200- computer Z603 

BT : Computers 

No scope note, NT, X, or SA. 

Create a new descriptor-. Circle- the letter- on the-answer sheet. 

A. Computers 

X Honeywell 1200 computer 2603 
Honeywell 1200 computer 2603 see Computers 



B, Computers 

BT Honeywell 1200 computer 



Honeywel 1 1 200 computer 2603 
NT Computers 



C. Computers 

NT Honeywell 1200 computer 2603 



D. 



Computers 

NT Honeywell 1200 computer 

Honeywell 1200 computer 2603 
BT Computers 
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Test 13 

Indexable information 6, JOVIAL programming language 
Step 5, Updating the SAL, 

Descriptor: JOVIAL 1768 

BT: Program languages 495 

No scope note, NT, X, or SA, 

Create a new descriptor-. Circle- the- letter-on- the-answer sheet, 

A, JOVIAL 1768 

SA Program languages 

Program languages 495 
SA JOVIAL 



B. JOVIAL 1768 

BT Program languages 

Program languages 495 
NT JOVIAL 



C, JOVIAL 1768 

BT languages 

Program languages 495 
NT JOVIAL 



D, Languages 328 

NT Program languages 495 
NT JOVIAL 1768 
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